COTS-Based Systems Top 10 List

I n the January 2001 issue of Computer (pp. 135-137), we published the Software Defect Reduction Top 10 List—one of two foci pursued by the National Science Foundation-sponsored Center for Empirically Based Software Engineering (CeBASE). COTS-based systems (CBS) provide the other CeBASE focus. For our intent, COTS software has the following characteristics: The buyer has no access to the source code; the vendor controls its development; and it has a nontrivial installed base (that is, more than one customer ; more than a few copies). Criteria for making the list are that each empirical result has • significant current and future impact on software dependability, timeli-ness, and cost; • diagnostic value with respect to cost-effective best practices; and • reasonable generality across applications domains, market sectors, and product sizes. These are the same criteria we used for our defect-reduction list, but they are harder to evaluate for CBS because it is a less mature area. CBS's roller-coaster ride along Gartner Group's visibility-maturity curve (http:// gartner11.gartnerweb.com/public/static/ hotc/hc00094769.html) reveals its relative immaturity as it progresses through a peak of inflated expectations (with many overenthusiastic organizational mandates to switch to CBS), a trough of disillusion-ment, and to a slope of enlightenment, to a plateau of productivity. We present the CBS Top 10 List as hypotheses, rather than results, that also serve as software challenges for enhancing our empirical understanding of CBS. More than 99 percent of all executing computer instructions come from COTS products. Each instruction passed a market test for value. • Source. The more than 99 percent figure derives from analyzing Department of Defense data (B. Boehm, " Managing Software Productivity and Reuse, " Computer, Sept. 1999, pp. 111-113). • Implications. Economic necessity drives extensive COTS use. Nobody can afford to write a general-purpose operating system or database management system. Every project should consider the CBS option, but carefully weigh CBS benefits, costs, and risks against other options. " Market test " means that someone willingly pays to install the COTS component, not that every instruction is used or proves valuable. More than half the features in large COTS software products go unused. working alone used 12 to 16 percent of Microsoft Word and PowerPoint measurement features, whereas a 10-person group used 26 to 29 percent of these features. • Implications. Adding features is an economic necessity for vendors but it introduces complexity for COTS adopters. This added complexity can require …